On the Certification of the Restricted Isometry Property 



o 

(N 



O 



Pascal Koiran* Anastasios Zouzias^ 

January 18, 2013 

Abstract 



Compressed sensing is a technique for finding sparse solutions to underdetcrmined linear sys- 
tems. This technique relies on properties of the sensing matrix such as the restricted isometry 
property. Sensing matrices that satisfy this property with optimal parameters are mainly ob- 
tained via probabilistic arguments. Given any matrix, deciding whether it satisfies the restricted 
isometry property is a non-trivial computational problem. In this paper, we give reductions from 
dense subgraph problems to the certification of the restricted isometry property. This gives ev- 
\ idence that certifying this property is unlikely to be feasible in polynomial-time. Moreover, 

on the positive side we propose an improvement on the brute-force enumeration algorithm for 
^ \ checking the restricted isometry property. 

• Another contribution of independent interest is a spectral algorithm for certifying that a ran- 

dom graph does not contain any dense fc-subgraph. This "skewed spectral algorithm" performs 
■ better than the basic spectral algorithm in a certain range of parameters. 

> 

00 1 1 Introduction 

G\ 

Let <3? be a n x N matrix with N > n. A vector x 6 is said to be /c-sparse if it has at most k 
nonzero coordinates. Given 5 G]0, 1[, <p is said to satisfy the Restricted Isometry Property (RIP) of 
order k with parameter 5 if it approximately preserves the Euclidean norm in the following sense: 
for every fc-sparse vector x, we have 

(l-<5)||x|| 2 < ||$x|| 2 < (l + 5)\\x\\ 2 . 

X ' 

Clearly, for this to be possible we must have k < n. Given 5, n and N, the goal is to construct RIP 
matrices with k as large as possible. This problem is motivated by its applications to compressed 
sensing: it is known from Candes, Romberg and Tao [8, 9, 10J that the restricted isometry property 
enables the efficient recovery of sparse signals using linear programming techniques. For that 
purpose one can take any fixed 5 < y/2 — 1 [8j. 

Various probabilistic models are known to generate random matrices that satisfy the RIP with 
a value of k which is (almost) linear n. See for instance Theorem 2 in Section [3] for the case 
of matrices with entries that are independent symmetric (±1) Bernouilli matrices. The recent 
survey [21j provides additional results of this type and extensive references to the probabilistic 
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literature. Some significant effort has been devoted to the construction of explicit (rather than 
probabilistic) RIP matrices, but this appears to be a difficult problem. As pointed out by Bourgain 
et al. in a recent paper [61 \7\, most of the known explicit constructions [TTJ [2j CE3] are based on 
the construction of systems of unit vectors with a small coherence parameter (see section [2] for a 
definition of this parameter and its connection to the RIP). Unfortunately, this method cannot 
produce RIP matrices of order k > yjn [61 \7\. Bourgain et al. still manage to break through the 
y/n "barrier" using techniques from additive combinatorics: they construct RIP matrices of order 
k = n l / 2+e ° where eo > is an unspecified "explicit constant" . Note that this is still far from the 
order achieved by probabilistic constructions. 

Here we study the restricted isometry property from the point of view of computational com- 
plexity: what is the complexity of deciding whether a matrix satisfies the RIP, and of computing or 
approximating its order k or its RIP parameter <5? An efficient (deterministic) algorithm would have 
applications to the construction of RIP matrices. One would draw a random matrix <3? from one 
of the well-established probabilistic models mentioned above, and run this hypothetical algorithm 
on $ to compute or approximate k and 5. The result would be a matrix with certified restricted 
isometry properties (see Section [3] for an actual result along those lines) . This may be the next best 
thing short of an explicit construction (and as mentioned above, the known explicit constructions 
are far from optimal). 

The definition of the restricted isometry property suggests an exhaustive search over sub- 
spaces, but we are not aware of any existing hardness result suggesting that exhaustive search is 
somehow unavoidable. There has been more work from the algorithm design side. In particular, 
it was shown that semi-definite programming can be used to verify the restricted isometry prop- 
erty [llj and other related properties from compressed sensing [12j[16]. Unfortunately, as pointed 
out in |12| these methods are unable to certify the restricted isometry property for k larger than 
0(i/n), even for matrices that satisfy the RIP up to order f2(n). As we have seen, k = 0{y/n) is 
also the range where coherence-based methods reach their limits. 

In this paper we provide both positive and negative results on the computational complexity of 
the RIP, including the range k > yjn. 

Positive Results 

In Section [2l we study the relation between the RIP parameters of different orders for a given matrix 
<£. Very roughly, we show in Theorem[T]that the RIP parameter is at most proportional to the order. 
We therefore have a trade-off between order and RIP parameter: in order to construct a matrix of 
given order and RIP parameter, it suffices to construct a matrix of lower order and smaller RIP 
parameter. We illustrate this point in Section [3l Our starting point is the above-mentioned (very 
naive) exhaustive search algorithm, which enumerates all (. ) subspaces generated by k column 
vectors. We obtain a "lazy algorithm" which enumerates instead all subspaces generated by I basis 
vectors for some I < k. We show that the lazy algorithm can go slightly beyond the y/n barrier if 
a quasi-polynomial running time is allowed. 

Negative Results 

We provide the first hardness results on the certification of the restricted isometry property. We 
could not base these results on a well-established assumption such as P ^ NP. Instead, we rely on 
certain assumptions on the complexity of dense subgraph problems. There is already a fairly large 
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literature on the complexity of detecting dense subgraphs in an input graph, with both algorithmic 
results and hardness results (see [5] for extensive references). In particular, it was shown by 
Khot [18] that the dense fc-subgraph problem does not have a polynomial time approximation 
scheme under the assumption that NP does not have randomized algorithms that run in sub- 
exponential time. Earlier, Feige [14] had obtained this result under the assumption that random 
3-SAT formulas are hard to refute. Our results are of a similar flavor as those of Feige [14] and 
Alekhnovich pQ since they are based on average-case assumptions. However, dense subgraphs do 
not appear at the end of the reduction as in [14] but at the beginning. Our starting point is 
the assumption that it is hard to certify that a random G(n, 1/2) graph does not contain any 
/c-subgraph with density at least 1/2 + e (note that 1/2 is the expected density). A certification 
algorithm must certify most of the graphs on n vertices, and any certified graph cannot contain 
any /c-subgraph with density at least 1/2 + e. More precisely, we present two distinct hardness 
hypotheses corresponding to different settings for the parameters k and e. In the first hypothesis 
we set k = n a and e = where a, /3 g]0, 1[ and a — (3 < 1/2. In the second hypothesis we 

set k = an and e = j/\/n where a €]0, 1[ is a small enough constant, 7 > and 07 is small 
enough. Here, "small enough" means that a and cry are both upper bounded by k, where k < 1 
is a universal constant. The second hypothesis postulates that with the above parameter settings, 
there is a value of n for which certifying most graphs on n vertices is computationally difficult. 

We believe that these two hypotheses are consistent with the current state of knowledge on 
dense subgraph problems. In particular, without the constraints on a, (3 and 7 these hypotheses 
would be refuted by a simple spectral algorithm. Each of the two hypotheses leads to different 
hardness results. From the first hypothesis it follows that RIP parameters cannot be approximated 
to within any constant factor in polynomial time. From the second hypothesis we derive the 
following stronger result: no polynomial time algorithm can distinguish between a matrix which 
satisfies the RIP of order k with parameter (say) k/1000 and a matrix which does not satisfy the 
RIP with parameter n/2 (here the constant 1000 can be replaced by any constant larger than 2). 

Finally, in an effort to further test the validity of these two hypotheses we propose a skewed 
spectral algorithm for certifying the absence of a fc-subgraph of density at least 1/2 + e in a random 
graph. This algorithm performs better than the basic spectral algorithm for certain settings of k 
and e, but is still consistent with our two hypotheses. 

Organization of the paper 

As explained above, the next two sections are devoted to positive results. In Section U] we work 
out some bounds on the eigenvalues of random matrices, for later use in our reductions from dense 
subgraph problems to the approximation of RIP parameters. We rely mainly on the classical 
work of Fiiredi and Komlos [15] as well as on a more recent concentration inequality due to Alon, 
Krivelevich and Vu [1] . In Section [5] we present our two hypotheses on the intractability of dense 
subgraph problems, and we show that they are consistent with what we know from the spectral 
algorithm. In Section we use these hypotheses to show that approximating RIP parameters is hard 
even for square matrices. In Section [8] we derive similar results for matrices of "strictly rectangular" 
format (which is the case of interest in compressed sensing). We proceed by reduction from the 
square case. Interestingly, this last reduction relies on the known constructions (deterministic [61 [7] 
and probabilistic [21]) of matrices with good RIP parameters mentioned earlier in the introduction. 
We therefore turn these positive results into negative results. The table at the end of Section [8] 
gives a summary of our hardness results. Section [6] is devoted to our skewed spectral algorithm. 
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This part of the paper can be read independently from the RIP results as it deals with a purely 
graph-theoretic problem. The analysis of the skewed algorithm is based on a new spectral norm 
estimate for a certain class of random matrices. 

2 Increasing the order by decreasing the RIP parameter 

As explained at the beginning of [HE], certain (suboptimal) constructions are based on the construc- 
tion of systems of unit vectors (ui, . . . ,ujsr) £ C n with small coherence. The coherence parameter 
\x is defined \(ui,Uj)\. Indeed, we have the following proposition. 

Proposition 1 Assume that the column vectors u%, . . . ,un of <3? are of norm 1 and coherence ji. 
Then satisfies the RIP of order k with parameter 5 = (k — 

We reproduce the proof from [6j [7] since if fits in one line: for any fe-sparse vector x, 

|||$x|| 2 - ||x|| 2 | < 2j2\xiXj(ui,uj}\ < M) 2 ~ INI 2 ) - ( fc ~ i)mII^II 2 - 

i<j i 

We now give a result, which (as we shall see) generalizes Proposition [TJ 

Theorem 1 Assume that $ has unit column vectors and satisfies the RIP of order m with param- 
eter e. For k > m, & also satisfies the RIP of order k with parameter 5 = e(k — l)/(m — 1). 

Proof. Let u±, . . . ,un be the column vectors of Let x be a fc-sparse vector, and write x = 
Yli^T x i u i wnere T is a subset of {1, ... , N} of size k. Since ||^a;|| 2 = | \x\ \ 2 + 2 J2i<j x i x j{ u ii u j)> 
to check the RIP of order k we need to show that 

| ^2xiXj(ui,Uj)\ < 5\\x\\ 2 /2, (1) 

i<j 

where 5 = e(k — l)/(m — 1). To estimate the left hand side, we compare it to the sum of the similar 
quantity taken over all subsets of size m of T, namely: 

I Yl XiXj{ui,Uj)\. (2) 

\S\=mi,jeS,i<j 

Since each pair (i, j) appears in exactly ( ~;>) subsets of size m, this sum is equal to [ m ^ 2 ) times 
the left-hand side of (H|). But we can also estimate (|2|) using the RIP of order m. For each subset 
S of size m, we have 

| ^2 XiXj(ui,Uj)\ < e^xf/2. 

This follows from ([1]), replacing 5 by e (the RIP parameter of order m). Since each term xf will 
appear exactly in (^~\) subsets, we obtain e( ~\) IM| 2 /2 as an upper bound for ([2]). We conclude 
that the left-hand side of © is bounded by §(£ll)|M| 2 /(£-a) = e^lMlV 2 - D 

We claim that Proposition Q] is the case m = 2 of Theorem [TJ This follows from the following 
observation. 
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Remark 1 For a matrix $ with unit column vectors, the coherence parameter ^ is equal to the 
RIP parameter of order 2. 

Proof. Let 5 be the RIP parameter of order 2. We have 5 < [i by Proposition [TJ It remains to show 
that S > fj,. Consider therefore two column vectors it, and Uj with \ (ui,Uj)\ = fj,. Let x = Ui + Uj. 
We have ||x|| 2 = 2 and ||$x|| 2 = 2 ± 2fi, so that S > \x indeed. □ 

3 A Matrix Certification Algorithm 

The naive algorithm for computing the RIP parameter of order k will involve the enumeration of 
the (^) submatrices of <i> made up of k column vectors of <£. For each T C {1,... , N} of size 
k let us denote by $r the corresponding n x k matrix. We need to compute (or upper bound) 
5 = maxT St, where 

6t = sup | ||$Ts|| 2 /lkl| 2 - 1 I- 

For each T, 5t can be computed efficiently by linear algebra. For instance, St is the spectral radius 
of the self-adjoint matrix <& t &t — Ifc- The cost of the computation is therefore dominated by the 
combinatorial factor i^T) due to the enumeration of all subsets of size k. 

Here we analyze what the naive algorithm can gain from Theorem [TJ We therefore consider the 
following lazy algorithm. The correctness of the algorithm follows immediately from Theorem [TJ 

Algorithm 1 
1: procedure Lazy(<3>, m, S) 

2: Input: a n x N matrix with unit column vectors, an integer m < n, and a parameter 
$€]0,1[. 

3: Compute as explained above the RIP parameter of order m. Call it e. 

4: Output: Certify f as a RIP matrix of order k with parameter S, for all k > m such that 

e(k-l)/(m-l) < S. 
5: end procedure 



We now analyze its behavior on random matrices, which are in many cases known to satisfy the 
RIP with high probability. Consider for instance the case of a matrix whose entries are independent 
symmetric Bernouilli random variables. 

Theorem 2 Let A be a nx N matrix whose entries are independent symmetric Bernouilli random 
variables and assume that n > Ce~ 2 mlog(eN / m) . With probability at least 1 — 2exp(— ce 2 n), the 
normalized matrix <1> = -j^A satisfies the the RIP of order m with parameter e. Here C and c are 
absolute constants. 

In fact the same theorem holds for a very large class of random matrix models, namely, subgaussian 
matrices with either independent rows or independent columns (|21j. Theorem 64). 

Proposition 2 Let A be a random matrix as in Theorem^ and S G]0, 1[. With probability at least 
1 — 2(eN/m)~ cCm , the lazy algorithm presented above will certify that A satisfies the RIP of order 
k with parameter S for all k such that: 

k<s x i : 

V clog{eJy/m) 
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Here c and C are the absolute constants from Theorem [H 

Proof. All parameters being fixed we take e as small as allowed by Theorem [21 so that ne 2 = 
Cmlog(eN/m). This yields the announced probability estimate, and the upper bound on k is 
6m/e. □ 

To compare the lazy algorithm to the naive algorithm, set for instance m = y/n. In applications 
to compressed sensing one can set 5 to a small constant value (any 5 < \[2 — 1 will do). Thus, 
disregarding constant and logarithmic factors, with high probability the lazy algorithm will certify 
the RIP property for k of order roughly n 3//4 . This is achieved by enumerating (1/2) subspaces, 
whereas the naive algorithm would enumerate roughly (Js/4) subspaces. 

Another choice of parameters in Proposition [2] shows that one can beat the y/n bound by a 
logarithmic factor with a quasi-polynomial time algorithm. For instance: 

Corollary 1 If we setm = (logAQ 3 , the lazy algorithm runs in time 2°( lo s 4Ar ) and, with probability 
at least 1 — 2~^ logAr ) ) certifies that A satisfies the RIP of order k with parameter 5 for all 
k < K5 log Ny/n, where K is an absolute constant. 

4 Eigenvalues of Random Symmetric Matrices 

Proposition [5] is the main probabilistic inequality that we derive in this section. It shows that 
square matrices obtained by Cholesky decomposition from a certain class of random matrices have 
good RIP parameters with high probability. This result is then used in Section [7] to give a reduction 
from dense subgraph problems to the approximation of RIP parameters. 

4.1 Model A 

Consider the following random matrix model: A is a symmetric k x k matrix with an = 0, and for 
i < j the independent symmetric Bernouilli random variables. 

Let Xi(A) > \2(A) > . . . Xk(A) be the eigenvalues of A. Let m s be the median of \ S (A). From 
the main result of [4] (bottom of p. 263) we have for t > the inequality: 

Pi[X s (A)-m s >t]< 2e~ t2 / 32s2 . 

From Fiiredi and Komlos ([IS], Theorem 2) we know that mi < 3ay/k for k large enough, where 
a = 1 is the standard deviation of the Ojj in the case i < j. Therefore we have 

Pr[Ai(A) > zVk + t] < 2e-* 2/32 . 

Since A^(j4) = — Ai(— A) and —A has same distribution as A, we also have 

Pr[A fc (^) < -3Vk -t}< 2e"* 2/32 

(one could also apply directly the bound on Xk(A) for the more general model considered in [2]). 
As a result: 

Proposition 3 There is an integer ko such that for all k > ko and for all t > we have: 

Pr[max|Ai(A)| > 3y/k + t] < 4e"* 2/32 . 

i 

Remark 2 The constant 3 in Proposition^ can be replaced by any constant bigger than 2 (see 
Theorem 2 in [15]). 
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4.2 Model B 



Next we consider the model where B is a symmetric kxk matrix satisfying the following condition: 
ba = 1, and bij = c ■ aij/y/n for i < j, where the are independent symmetric Bernouilli random 
variables. Here c > is a fixed constant, and n is an additional parameter which should be thought 
of as going to infinity with k. 

Corollary 2 Assume that k > ko and that 5y/n > 3cV~k. Then the eigenvalues of B all lie in the 
interval [1 — 5, 1 + 5] with probability at least 



l-4exp[-(^-3v^) 2 /32]. 



Proof. We have B = + cA/y/n, where A follows the model of Proposition [3j The result therefore 
follows from that proposition by choosing t so that c(3yk + t)/y/n = 5, i.e., t = 5yfnjc — 3\/k. □ 

In the next corollary we look at the case n = k of this model. 

Corollary 3 Assume that n > ko and 3c < 1. Then B is positive semi-definite with probability at 
least 

l-4exp[-(l/c-3) 2 n/32]. 
Proof. Set n = k and 5 = 1 in Corollary [2j □ 

In the last result of this subsection we consider again the model B = l n + cA/y/n. Given a n x n 
matrix M and two subsets S, T C {1, , . , , n} of size k, let us denote by M$ y T the kxk sub-matrix 
made up of all entries of M of row number in S and column number in T. 

Corollary 4 Consider the random matrix B = I n + cA/y/n where A is drawn from the uniform 
distribution on the set n x n symmetric matrices with null diagonal entries and ±1 off-diagonal 
entries. 

If n> k> ko, then with probability at least 



1 — 4 exp 



k ln(ne/fe) 



- 3^/32' 



the submatrices B$,s have all their eigenvalues in the interval [1 — 5,1 + 5] for all subsets S C 
{1, . . . , n} of size k. 

Proof. By Corollary O for each fixed S matrix B$ t s has an eigenvalue outside of the interval 
[1 — 5, 1 + 5] with probability at most 4exp[— — 3\/^) 2 /32]. The result follows by taking a 
union bound over the (?) < (ne/k) k subsets of size k. □ 



4.3 Model C 

In Corollaries [3] and U] we considered the following random model for B: set B = I n + cA/y/n, 
where A is chosen from the uniform distribution on the set S n of all symmetric matrices with null 
diagonal entries and ±1 off-diagonal entries. If B is positive semi-definite, we can find by Cholesky 
decomposition a n x n matrix C such that C T C = B. If B is not positive semi-definite, we set 
C = 0. This is the random model for C that we study in this subsection. 
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Proposition 4 Assume that n > k > ko and that 3c < mm(l,5y/n/y/k). With probability at least 



1 — 4 exp 



k \n(ne/k) 



3\/fc) 2 /32 



4exp[-(l/c-3) 2 n/32], 



C satisfies the RIP of order k with parameter 8. 



Proof. If B = I n + cA/y/n is not positive semi-definite then C = and this matrix obviously does 
not satisfy the RIP. By Corollary El B can fail to be positive semi-definite with probability at most 
4exp[-(l/c-3) 2 n/32]. 

If B is positive semi-definite then C T C = B. Using the notation of Corollary EJ matrix C 
satisfies the RIP of order k with parameter 5 if for all subsets S of size k, the eigenvalues of the 
k x k matrices (C T C)s,s all lie in the interval [1 — <5, 1 + S] . Since C T C = B, by Corollary H this 
can happen with probability at most 4exp[/cln(ne//c) — — 3\/k) 2 /32]. □ 



5 Dense Subgraph Problems 

In this section we formulate precisely our two hypotheses on the intractability of dense subgraph 
problems. We show that they are compatible with what follows from the spectral algorithm (see 
Theorem [3] below). Spectral methods have become classical in graph theory, see for instance [3j for 
a nontrivial application to the detection of large hidden cliques in random graphs. 

We work with the standard G(n, 1/2) random graph model: each of the n(n — l)/2 potential 
edges is chosen with probability 1/2, and these choices are made independently. 

Definition 1 We say that a graph H on k vertices has density A if it has at least Xk(k — l)/2 
edges. For e > 0, we say that G has excess e if it has density 1/2 + e. 

Lemma 1 In a random graph on n vertices, the probability of existence of a k-subgraph with excess 
e is upper bounded by 

exp[/cln(ne/£;) — e 2 k(k — 1)]. 

Proof. Fix any subgraph H on k vertices. The number of edges in H is a sum of k(k — l)/2 
independent Bernouilli random variables. Therefore, by Hoeffding's inequality the probability for 
H to have excess e can be upper bounded by exp[— e 2 k(k — 1)]. The result follows by taking a union 
bound over the (?) < (ne/k) k subgraphs of size k. □ 

This probability goes rapidly to once e becomes significantly larger than \j\fk. For instance: 

Corollary 5 Fix any constant c > 0. For e > (1 + c)y/ln(ne/k)/k, the probability that a random 
graph on n vertices has a k-subgraph with excess e goes to as k,n —> +oo. 

When e is in the range of Corollary it makes sense to consider algorithm which will certify that 
most graphs on n vertices do not have any fc-subgraph with excess e (i.e., a certified graph should 
not have any /c-subgraph with excess e, and most graphs should be certified). 

As we shall now see (in Theorem [3]), the spectral method provides an efficient certification 
algorithm if ekj 'y/n is sufficiently large. 
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Lemma 2 Let A be the signed adjacency matrix of a graph G.IfG has a k-subgraph H with excess 
e then there is a unit vector x G M. n supported by k basis vectors such that x T Ax > 2e(k — 1). 

Proof. Assume without loss of generality that V(G) = {1, ... ,n} and V(H) = {1, ... ,&}. We 
have an = and for i ^ j, = 1 if ij E E; aij = — 1 if ij^E. Let x E K n be the unit vector 
uniformly supported by the first k basis vectors: Xi = \j\fk for i < k, xi = for i > k. Then 
x T Ax = 2j2 1 < l<J < k a ij /k>2e(k-l). □ 

Corollary 6 Under the same hypothesis as in Lemma [U the largest eigenvalue of A satisfies 
Xi(A) > 2e(k-l). 

Proof. This follows directly from the lemma since for a symmetric matrix, Ai(vl) = supiMi—i x 7 Ax. 
□ 

This leads to the following polynomial-time certification algorithm. 

Algorithm 2 
1: procedure Spectral(G) 
2: Input: a graph G on n vertices. 

3: Compute \\{A), where A is the signed adjacency matrix of G. 

4: Output: Certify that G does not contain any fe-subgraph with excess e for all k, e such that 

2e(k-l) > Xi(A). 
5: end procedure 



Theorem 3 The spectral algorithm presented above certifies that most graphs on n vertices do not 
contain any k-subgraph with excess e for all k,e such that 

2e(k - 1) > 3V^. 

Proof. The correctness of the algorithm follows from Corollary [6J By Theorem 2 in [15] most 
graphs satisfy Ai(^4) < so most graphs are certified. □ 

Remark 3 The constant 3 in this theorem can be replaced by any constant larger than 2 (see also 
Remark^). 

In the case e = 1/2, the spectral algorithm of Theorem [3] certifies that most graphs on n vertices 
do not contain any fc-clique for k > 1 + 3-^/n. 
We put forward the following hypotheses. 

Hypothesis 1 Fix two arbitrary rational constants a,/3 g]0, 1[ such that a — (3 < 1/2. Set e(n) = 
1/n^ and k(n) = n a (we only consider values of n such that k{n) G Nj. 

No polynomial time algorithm can certify that most graphs on n vertices do not have any k(n)- 
subgraph with excess e(n). 

Hypothesis 2 There is a constant k G]0, 1[ such that the following holds: fix two arbitrary con- 
stants a g]0, 1[ and 7 > with a < k and a^f < k. Set e(n) = j/'^/n and k(n) = an (again we 
only consider values of n such that k(n) G N). Then no polynomial time algorithm can certify that 
most graphs on n vertices do not have any k{n)-subgraph with excess e(n). 
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The constraints on the parameters a, (3, 7 in these two hypotheses are meant to ensure consistency 
with what we know from the spectral algorithm: in Hypothesis Q] we have ek = n a ~^ < \fn. In 
Hypothesis [2] we have ek = ary/y/n < Kyjn < y/n. By Remark [31 if we allowed a value n > 1 
in Hypothesis [2] then this hypothesis would become provably incorrect. Hypothesis [2] essentially 
means that the spectral algorithm can be improved at most by a constant factor. As we will see 
later, both hypotheses are also consistent with the skewed algorithm of Section El 

Of course, our two hypotheses are meaningful only for values of the parameters for which it is 
actually the case that most graphs do not have a A:-subgraph with excess e (otherwise, they are 
vacuously true). By Corollary [5l for the first hypothesis it is enough to have f3 < a/2. Therefore 
we can first pick any j3 G]0, l/2[ and then any a G]2/3, [3 + l/2[ (the upper bound /3 + 1/2 enforces 
the constraint a — /3 < 1/2 in Hypothesis H]). 

Let us now work out the corresponding constraints for Hypothesis [2) Take for instance c = 1 
in Corollary [5j Plugging in k = an and e = j/\/n yields the condition 

ln(e/a) 
7>2a/^^. 
V a 

But we also have the constraint 7 < n/a from Hypothesis [2J So we should have 

n/a > 2l/S3. 
V a 

This constraint is satisfied by all sufficiently small a. Hence we can first pick any small enough a, 
and then any 7 in the interval [2-y/ln(e/ 'a) / 'a, n/a]. 



6 A Skewed Spectral Algorithm 

Let G be an undirected graph on n vertices and A its signed adjacency matrix. We define the 
skewed adjacency matrix of G by the formula 

A=^ + ^3, (3) 

2 Jn 



where J is the n x n all l's matrix, and a > a parameter to be tuned later. We call the matrix 
A skewed because its entries are random variables with mean slightly above zero. 

Lemma 3 If G has a k-subgraph H with excess e then there is a unit vector x £ W 1 supported by 
k basis vectors such that x 1 Ax > e(k — 1) + kaj \fn. 

Proof. Assume without loss of generality that V{G) = {l,...,n} and V(H) = {1, . . . ,k}. Let 
x G M 71 be the unit vector uniformly supported by the first k basis vectors: X{ = 1/y/k for i < k, 
Xi = for i > k. We have seen that x 1 Ax > 2e(k — 1). The conclusion follows since x T 3x = k. □ 

This simple lemma forms the basis for the following skewed algorithm. In the case a = 0, this is just 
the standard spectral algorithm. Since A is symmetric, Xi{A) = sup|| x .|| =1 x T Ax. The correctness of 
the skewed algorithm therefore follows from Lemma [3l In the remainder of this section, we analyze 
the behavior of this algorithm on random graphs. 
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Algorithm 3 



1: procedure Skewed Spectral(G, a) 
2: Input: a graph G on n vertices. 

3: Compute Ai(j4), where as above A is the skewed adjacency matrix of G. 

4: Output: Certify that G does not contain any /c-subgraph with excess e for all k, e such that 

e(k - 1) + kajyfn > \\{A). 
5: end procedure 



Theorem 4 Let G be a random graph on n vertices and A its skewed adjacency matrix as defined 



by ([3p. The expectation of the operator norm 



A 



su P||x|| = l 



Ax 



satisfies the inequality 



E 



A 



< v^Va 2 + 5/4 + o(l). 



Moreover, for all e > we have the concentration inequality 



Pr 



A 



> E 



A 



+ e 



< exp(-e 2 /8). 



We defer the proof of the above theorem to the appendix. Compare this bound on E 



A 



with the 



bound (a + l + o(l))y / n that follows from applying the triangle inequality on A. A direct calculation 
shows that the bound in Theorem is better for a > 1/8. 

For any matrix, the largest eigenvalue is upper bounded by the operator norm. We therefore 
have the following corollary. 

Corollary 7 For most graphs G on n vertices, the skewed adjacency matrix of G satisfies the 

inequality 

Ai(l) < vW« 2 + V4 + o(l). 

Now, let k and e be two functions of n. From Corollary [7] and the correctness of the skewed 
algorithm it follows that whenever the inequality 



lim inf 

hoo s/n 



e(n).(k{n) - 1) 



ak(n) 



> ^a? + 5/4 



(4) 



is satisfied, the skewed algorithm can certify that most graphs on n vertices do not have any 
fc(n)-subgraph with excess e(n). In particular, we have obtained the following result. 

Theorem 5 Set e(n) = "f/y/n and k(n) = an, where a g]0, 1[ and 7 > are two constants. If 
cry + aa > \J a? + 5/4 then the skewed algorithm can certify that most graphs on n vertices do not 
have any k(n) -subgraph with excess e(n). 

For instance, taking a = 1 yields the following. 

Corollary 8 // 07 + a> 3/2 then the skewed algorithm can certify that most graphs on n vertices 
do not have any an-subgraph with excess j/^/n. 
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This is clearly an improvement over the standard spectral algorithm, since that algorithm is unable 
to certify that most graphs on n vertices do not have any an-subgraph with excess j/y/n when 
«7 < 1. But there are values a e]0, 1[ and 7 > for which 07 < 1 and the condition of Corollary [8] 
is satisfied (take for instance a = 3/4 and 7 = 5/4). 

More generally, given values of a and 7 we can ask when there is a value of a such that the 
skewed algorithm can certify that most graphs on n vertices do not have any an-subgraph with 
excess 'y/y/n. 

Corollary 9 // the condition 4a 2 7 2 + 5a 2 > 5 is satisfied, the skewed algorithm can certify that 
most graphs on n vertices do not have any an-subgraph with excess ^/y/n. 

Proof. In view of Theorem [51 we are looking for a value of a such that 

f{a) = a 2 + 5/4 - (cry + aaf < 0. 

There is such an a if A > 0, where A is the discriminant of the quadratic polynomial f(a). But 

A = 4aV - 4(1 - a 2 )(5/4 - aV) = -5 + 4aV + 5a 2 . 

□ 



Note that this analysis of the skewed algorithm is suboptimal: in the case a = we can certify 
with the standard spectral algorithm that most graphs on n vertices do not have any an-subgraph 
with excess j/y/n as soon as 07 > 1. But this fact cannot be derived from Corollary [9l In order to 

improve this corollary, one would need to improve the upper bound on E A given by Theorem SJ 
It follows from Corollary [9] that the constant k in Hypothesis [2] must satisfy the constraint 
9k 2 < 5, i.e., k < y/E/3 ~ 0.745. 

Finally, let us consider the case where (as in Hypothesis [1]) the two functions k(n) and e(n) are 
of the form k(n) = n a and e(n) = Xjnr for two constants a,/3 G]0, 1[. With those settings, the 
liminf in ^ is equal to if a — (3 < 1/2, to 1 if a — f3 = 1/2 and to +00 if a — ft > 1/2. Hence 
we do not obtain any improvement over the standard spectral algorithm (both algorithms succeed 
only in the case a — /3 > 1/2). 



7 Dense Subgraphs and the Restricted Isometry Property 

In this section we show (in Theorems [6] and [TJ that RIP parameters are hard to approximate even 
for square matrices. We establish connections between dense subgraphs problems and the RIP 
thanks to a generic reduction which we call the Cholesky reduction. This reduction maps a graph 
G on n vertices to a n x n matrix C(G). Let A be the signed adjacency matrix of G: we have 
an = and for i ^ j, aij = 1 if ij E E; aij = — 1 if ij^E. We construct C = C{G) from A 
using the procedure described in Section 1431 That is, we first compute B = I n + cAj^fn (suitable 
choices for the parameter c will be discussed later). If B is not positive semi-definite, we set C = 0. 
Otherwise, we find by Cholesky decomposition a matrix C such that C T C = B. 

For suitable values of k, C(G) satisfies the RIP of order k for most graphs G. This was made 
precise in Proposition [H On the other hand, we shall see that if G has a A:-subgraph that is too 
dense then C{G) does not satisfy the RIP of order k. 
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Proposition 5 Let G be a graph on n vertices and C(G) its image by the Cholesky reduction. If 
G has a k-subgraph with excess e and 

5 < 2ce{k - 1)/Vn 
then C(G) does not satisfy the RIP of order k with parameter 5. 

Proof. Let A be the signed adjacency matrix of G and B = I n + cA/y/n. If B is not semi-definite 
positive, C(G) = does not satisfy the RIP. Otherwise C T C = B. Let x be the vector of Lemma[2j 
We have ||Ca;|| 2 = x T C T Cx = x T Bx = 1 + cx T Ax/y/n~ >l + 8. □ 

7.1 Hardness from Hypothesis [j] 

Lemma 4 Set k = n a and e = \jn^ where (as suggested in Section^ a, ft are two constants 
satisfying the constraints ft e]0, l/2[ and a g]2/3,/3 + l/2[. Set also 5 = c'ek/y/n where c' > is 
another constant. Then for most graphs G onn vertices the matrix C(G) satisfies the RIP of order 
k with parameter 5. 

Proof. Fix a constant c < 1/3. We can apply Proposition[3]since the hypothesis 3c < min(l, b\Jn/\fk) 
is satisfied for n large enough (note that b\fn/\fk = c'eVk = c'n a ^ 2 ~^ , and the exponent a/2 — j3 
is positive). 

In Proposition [4J the probability that C = C (G) does not satisfy the RIP is bounded by a sum 
of two terms. The second one, 4exp [-(1/c- 3) 2 n/32], clearly goes to as n — > +oo. To obtain 
the same property for the first term we consider the argument: 

k hx(ne/k) - (^p - 3^) 2 /32. (5) 

Note that 5^/n/c = {d /c)ek = (c' /c)n a ~@ , so that this term dominates the term 3\/k = 3?W 2 since 
a — ft > a/2. Moreover (Sy/n/c) 2 = (c ; /c) 2 n 2 ( a ~^ dominates the term k1n(ne/k) since k = n a and 
2(a — ft) > a. Therefore ([5]) is equivalent to — (c'/c) 2 n 2 ( a_ ' 3 )/32, and its exponential goes to as 
n goes to infinity. Hence the same is true for the sum of the two error terms. □ 

Theorem 6 Under Hypothesis^ no polynomial-time algorithm can approximate the RIP param- 
eters of square matrices within any constant factor. 

More precisely, fix any constants A > 1 and 5q < 1/A. Under Hypothesis [1\ no polynomial time 
algorithm can distinguish between matrices which satisfy the RIP of order k with parameter 5 < 5q 
and matrices which do not satisfy the RIP of order k with parameter XS. 

In this theorem a distinguishing algorithm must accept all inputs which satisfy the RIP of order k 
with parameter 5 < 5q, and reject all inputs which do not satisfy the RIP of order k with parameter 
XS. Its behavior on other inputs is not specified. 

Proof of Theorem® The second part of the theorem clearly implies the first: if we have an algorithm 
which approximates RIP parameters within some factor 7 > 1 then we have a distinguishing 
algorithm with A = 7 2 . 

Let us therefore assume that we have a distinguishing algorithm A with the associated constants 
A and 5q. Fix a positive constant c < 1/3 and two constants a and ft satisfying the constraints 
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of Lemma [U for instance /3 = 1/3 and a = 3/4. Set as usual k = n a and e = . We will use 
A to certify that most graphs on n vertices do not have any /c-subgraph with excess e, thereby 
contradicting Hypothesis [TJ 

On the one hand, by Lemma |4] for most graphs G the matrix C(G) satisfy the RIP of order k 
with parameter 5 = dek/y/n (we will see quite soon that d can be any sufficiently small constant). 
In this case 5 = dek/y/n = dn a ~^~ 1 / 2 goes to as n — > +oo, so for n large enough we have 5 < 5q 
and we can use the hypothesis on our distinguishing algorithm. 

On the other hand, if G has a /c-subgraph with excess e then by Proposition[5]the RIP parameter 
of C(G) is not smaller than 2ce(k — l)/y/n. This is more than X5 if Ac' < 2c and n is large enough. 
Therefore given A and c we choose a constant d < 2c/ X, and by running A on C(G) we can certify 
that most graphs on n vertices do not have any fe-subgraph with excess e (we certify G if A accepts 
C{G)). □ 

Note that the above hardness result has been established for very small RIP parameters: the 
matrices in the proof have RIP parameters of order ek/y/n = n a ~^" 1 '' 2 and the exponent a — (3 — 1/2 
is negative. This may be viewed as a weakness of the result since applications to compressed sensing 
only require a constant 6 (as pointed out in the introduction, any 5 < V2— 1 will do). We partially 
overcome this weakness in Section E21 but for this we need to replace Hypothesis [T] by Hypothesis [2j 
Note also that the proof of Theorem [6] yields more information than contained in the statement 
of the theorem. For instance, with the settings a = 3/4 and ^ = 1/3 we see that the gap problem 
in Theorem [6] remains hard even for k = n 3 / 4 and 5 = c'/n 1 / 12 . We have included this additional 
information in the table at the end of the paper, for Theorem [6] as well as for our other hardness 
results. 

7.2 Hardness from Hypothesis [2] 

Lemma 5 Set k = an and e = r y/\/n where a G]0, 1[ and 7 > are two constants. Fix a constant 
c < 1/3 and another constant d such that 3c < d^^fa and 

ln(e/a) < (^^ - 3) 2 /32. (6) 

c 

Finally, set 5 = dek/y/n = c'a^. Then for most graphs G on n vertices the matrix C(G) satisfies 
the RIP of order k with parameter 5. 

Proof. We can apply Proposition[3]since the hypothesis 3c < min(l, 5yfn/ Vk) is satisfied: 5^/n/Vk = 
5/y/a = d^^fa > 3c by choice of d . 

As in the proof of LemmaHl the second error term in Proposition 0] clearly goes to as n — > +00. 
We therefore turn our attention to the first error term, and in particular to the argument © of 
the exponential function. 

In © the term b~\fnjc — 2>Vk is equal to (c'aj/c — 3yfa)y/n so the term — 3\/fc) 2 /32 is 
equal to aid ^faAjc — 3) 2 ra/32. 

The positive term in © is kln(ne/k) = anln(e/a). Overall, ([5]) is equal to —Kn for some 
constant K, and K is positive by ©. □ 

Theorem 7 Fix any constant 5q < n/2, where k is the constant from Hypothesis^ Assuming this 
hypothesis, no polynomial time algorithm can distinguish between matrices which satisfy the RIP 
of order k with parameter So and matrices which do not satisfy the RIP of order k with parameter 
k/2. 
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Proof. Let us assume the contrary, and let A be the distinguishing algorithm. Set k = an and 
6 = 7 / /y / n where 7 = n/a and the constant a e]0, k[ is small enough (we will see at the end of the 
proof how small is small enough) . We will use A to certify that most graphs on n vertices do not 
have any fc-subgraph with excess e, thereby contradicting Hypothesis [2j 

Consider first the case of a graph G having a fc-subgraph with excess e. Let us fix a constant 
c < 1/3 such that 2cn > k/2. Then by Proposition [5] the matrix C(G) does not satisfy the RIP of 
order k with parameter k/2. 

On the other hand, if we can find a constant d which satisfies the hypotheses of Lemma [5] and 
such that 5 = daj = kc' < 5q then C(G) will satisfy the RIP of order k with parameter 5o for most 
graph G. We will therefore be able to use A to certify that most graphs on n vertices do not have 
any /c-subgraph with excess e. 

It just remains to explain how to choose d . We will simply pick any d such that kc' < 5q and 
will then check that the hypotheses of Lemma [5] are satisfied if a is small enough. First we have 
to check the condition 3c < d^fy/a. The right hand side is equal to kc' / y/a and will exceed the 
left-hand side if e.g. y/a < d 'k (recall that 3c < 1). The other condition to be checked is ([6]). 
For the same reason, it will be satisfied for small enough a: the right-hand side is equivalent to 
(kc' / c) 2 / (32a) , which dominates the logarithmic function on the left-hand side. □ 

8 Hardness for Rectangular Matrices and Randomized Certifica- 



In this section we show that the RIP parameters of rectangular matrices are hard to approximate. 
This is the case of interest in compressed sensing. In a sense this was already done in Section [7) we 
have shown that the special case of square matrices is already hard. Nevertheless, it is of interest to 
know that the problems remains hard for strictly rectangular matrices. This is what we do in this 
section. Proofs are essentially by reduction from the square case. We begin with a simple lemma. 

Lemma 6 Consider a matrix $ with the following block structure: 



This matrix satisfies the RIP of order k with parameter 5 if and only if the same is true for both 
A and B. 

Proof. For an input vector x with the corresponding block structure x = (u v) we have ||x|| 2 = 
||ii 2 || + |M| 2 an d ll^ 2 -!! 2 = ll^^ll 2 + ll-^ll 2 - Therefore, if <3? satisfies the RIP of order k with 
parameter 5 then the same is true for A (take v = and u fc-sparse). The same argument applies 
also to B. 

Conversely, assume that A and B satisfy the RIP of order k with parameter 5. Let x = (u v) 
be a /c-sparse vector. We have ||<&x|| 2 — ||x|| 2 = (||3>ii|| 2 — ||u|| 2 ) + (||<I>u|| 2 — ||f|| 2 ). Both u and v 
must be /c-sparse, so the first term is bounded in absolute value by <5||ii|| 2 and the second one by 
<5||f|| 2 . The result follows since ||n|| 2 + \\v\\ 2 = ||£|| 2 . □ 

Theorem 8 Fix any constants A > 1 and 60 < 1/A. Under Hypothesis [7J no polynomial time 
algorithm can distinguish between matrices which satisfy the RIP of order k with parameter 5 < 5o 
and matrices which do not satisfy the RIP of order k with parameter X5. 



tion 
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Moreover, polynomial-time distinction between these two cases remains impossible even for ma- 
trices of size In x (n + N) where N = n 1+e °. Here €q g]0, l/2[ is an absolute constant (independent 
of A and 60 ). 

The first part of this theorem is just Theorem [6j The second part is new. Its proof is a refinement 
of the proof of Theorem [6] (and refers to it). 

Proof of Theorem From a graph G on n vertices we construct the matrix 



where C(G) is as in the previous section and B n is a matrix with good restricted isometry properties. 
Its role is to ensure the rectangular format that we need for C'(G). Our specific choice for B n is the 
matrix constructed in @[7]- It is of size n x N where N = n 1+€ °, and it satisfies the RIP of order 
n2 +e ° with parameter n~ e °. Moreover, B n can be constructed deterministically in time polynomial 
in n. Note that C'{G) is of size 2n x (n + N) as required in the statement of Theorem El 

Let us assume that we have a distinguishing algorithm A which works for matrices of size 
2n x (n + N). We will certify that a graph G on n vertices does not have any /c-subgraph with 
excess e if _A accepts C'(G). As in the proof of Theorem [6j for suitable choices of k, e and 5 this 
will yield a contradiction with Hypothesis [TJ We now fill in the remaining details. 

Instead of setting a = 3/4 and j3 = 1/3 we now set a = eo; in order to satisfy the constraints of 
Lemma0]we need to take f3 in the interval ]eo, + e o)[- F° r reasons that will soon become clear 
we pick a f3 which also satisfies the additional constraint ft < 2eg, and we set as usual k = n a and 
e = 1/n^. It remains to show that we can use A to certify that most graphs on n vertices do not 
have any /c-subgraph with excess e. 

On the one hand, as explained in the proof of Theorem [6] for most graphs C{G) satisfies the RIP 
of order k with parameter 5 = c'n a ~^~ 1 / 2 (the constant c' is chosen as in that proof). Remember 
also that B n satisfies the RIP of order k with parameter n~ e °. Due to the choice (3 < 2cq we have 
n~ £ ° < 5 for n large enough, so by Lemma [6] for most graphs G the matrix C'(G) will satisfy the 
RIP of order k with parameter 8. 

On the other hand, we have seen in the proof of Theorem [6] that if n is large enough and G has 
a A;-subgraph with excess e then its RIP parameter is larger than X5. By Lemma [6] the same is true 



We can also give a hardness result for rectangular matrices based on Hypothesis [2j In fact we need 
a randomized version of this hypothesis which we state explicitly below. 

Hypothesis 3 (Randomized version of Hypothesis 2) There is a constant k g]0, 1[ such that 
the following holds. 

Fix two arbitrary constants a E]0, 1[ and 7 > with a < k and 07 < k. Set e(n) = j/y^n 
and k(n) = an (again we only consider values of n such that k(n) G Nj. Then no polynomial time 
randomized algorithm can certify that most graphs on n vertices do not have any k{n)-subgraph 
with excess e{n). 

In the above hypothesis, the hypothetical randomized algorithm would have to satisfy the following 
properties: 




of C 



(G). □ 
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(i) For most graphs G on n vertices, with probability at least (say) 3/4 the algorithm certifies 
that G does not contain any fc(n)-subgraph with excess e(n). 

(ii) For all graphs G on n vertices, if G is certified then it is always true (with probability 1) that 
G does not contain any A:(n)-subgraph with excess e(n). 

Note that the probability bounds in (i) and (ii) refer to the internal coin tosses of the algorithm. 

Theorem 9 Fix any constant 8q < k/2, where k is the constant from Hypothesis^ Assuming this 
hypothesis, no polynomial time algorithm can distinguish between matrices which satisfy the RIP 
of order k with parameter 5q and matrices which do not satisfy the RIP of order k with parameter 



Moreover, polynomial-time distinction between these two cases remains impossible even for ma- 
trices of size 2n x lOOn. 

Again, only the second part of the theorem is new. 

Proof. As in the proof of Theorem [8] we construct from a graph G a matrix of the form 



For B n , instead of of the deterministic construction from [6l[7] we will use a n x 99n random matrix 
given by Theorem [2j As before, we will certify that G does not have any fc-subgraph with excess 
e if the hypothetical distinguishing algorithm accepts C'(G). This will yield a contradiction with 
the randomized version of Hypothesis EJ 

In the proof of Theorem [7] we showed how to set the various parameters so that C (G) does not 
satisfy the RIP of order k with parameter 1/2 in the case where G has a /c-subgraph with excess 
e. In this case, by Lemma [6] it is also true that C'(G) does not satisfy the RIP of order k with 
parameter 1/2. Therefore we will never certify a graph which contains a /c-subgraph with excess e 
(note that this holds irrespective of the choice of B n ). 

We now turn our attention to the other case. By Theorem [21 with probability approaching 1 
as n — > +oo matrix B n will satisfy the RIP of order k with parameter 5q if the condition 



is satisfied. Since k = an and N = 99n this is equivalent to 5q > Ca log(99e/a). Note that the 
right-hand side of this expression goes to with a, but the left-hand side is constant. Condition [7] 
will therefore be satisfied for any small enough a. But recall from the proof of Theorem [7J that for 
small enough a we can set the other parameters so that C{G) satisfies the RIP of order k with 
parameter 5q for most graphs G. With such a setting of parameters, we conclude from Lemma [6] 
that C'(G) satisfies the RIP of order k with parameter 5o for most graphs and most choices of the 
random matrix B n . □ 

The constant 100 in Theorem[9]can be replaced by any other constant. Note also that the hypothet- 
ical polynomial-time algorithm in this theorem remains deterministic: it is only the (hypothetical) 
algorithm for graph certification which is randomized. It is clear, however, that Theorem [9] remains 
true for randomized algorithms with one-sided error. 



At/2. 




n > C5Q 2 klog(eN/k). 



(7) 
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Note that depending on the value of the constant k in Hypothesis O Theorem [9] does not rule 
necessarily rule out the existence of a polynomial time algorithm for deciding whether a RIP param- 
eter is smaller than y2 — 1 (recall from the introduction that this is good enough for applications 
to compressed sensing). Such an algorithm is ruled out if y/2 — 1 g]k/2, k[, but could in principle 
exist if k < y/2 - 1. @f 

Likewise, our hardness results do not rule out the existence of a polynomial-time algorithm dis- 
tinguishing between matrices with a very small RIP parameter and matrices with a RIP parameter 
larger than say 0.1. Here very small means as in Theorem [6] that the RIP parameter goes to as 
n — > +oo. If convergence to is not too fast then we could still use such a weak distinguishing al- 
gorithm for certifying most random matrices. The following table gives a summary of our hardness 
results. 



Hardness Results 


k 


(k, #i) vs. (k, S%) - hard 


Result 


Assumptions 


Dimensions (n x A^) 


n 3/4 


S 1 = cm' 1 ' 12 , 5 2 = Xcm- 1 ' 12 
VA > 1 


Theorem [6] 


Hypothesis Q] 


n X n 


„l/3 


Si = an- 4 ' 15 , S 2 = Acin- 4 / 15 
VA > 1 


Theorem [6] 


Hypothesis [T] 


n x n 


6(n) 


Si = K/2, S 2 = K 


Theorem [7] 


Hypothesis [2] 


n x n 




5 1 = c 2 n- 1 /i2 j S2 = Acan-V" 
VA > 1 


Theorem 1 


Hypothesis Q] 


n x n 1+e ° for e > 


0(n) 


Si = k/2, 5 2 = k 


Theorem [9] 


Hypothesis [3] 


n x 50n 



Table 1: We say that a matrix $ has the (k,5)-RIP iff (1-5) < \\&x\\ 2 < (1 + 5) for every k- 
sparse unit vector x. By (k, Si) vs. (/c, 52)-fiard we abbreviate the following: no polynomial time 
algorithm can distinguish matrices $ that satisfy the (k, <5i)-RIP from matrices that do not satisfy 
the (k, (52)-RIP- The constants c\,ci depend on A. 
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Appendix 

Proof of Theorem [4] We recall a bound on the expectation of a symmetric Bernoulli matrix \22\ 
Eqn. 1]. 

Lemma 7 Let A be a symmetric matrix whose off-diagonal entries are i.i.d. from {±1} with equal 
probability and having zero diagonal entries. There exist positive constants 5, c\ and C2 so that for 
k = c\n & 

Etr(A k ) < c 2 n(2^) fc . (8) 



By definition of the operator norm, we know that \\A\\ := supiM^ \\Ax\\ for any matrix A. Decom- 
pose any unit vector as x = al/y/n + f3y, where 1 is the all-ones vector, y is the unit vector with 
y_Ll and a 2 + /3 2 = 1. 
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where we used the triangle's inequality, Cauchy-Schwarz's inequality, Jensen's inequality and the 
fact that Jy = for every y_Ll. Now we bound each of the two terms on the right-hand side of ([9]) 
separately; the first term is straight-forward to bound. Indeed, 
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Now, take expectation and recall that KAij = and EA|- = 1. Since the entries of A are indepen- 



2 



dent, we obtain the inequality: 

- 1 2 

E A— < (a 2 + l/4)n. (10) 
\M 

To bound the second term, we use Wigner's trace method (Lemma|7f. For k as in Lemma [3 

|2 



EIUI 
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A 2k \\ < EVtr(A 2k ) < {Etr(A 2k ) < ± n 2°^ n )/ n ). 



where we used that ||^4 2fc || < tr(^4 2fc ), Jensen's inequality and Lemma [71 The above discussion 
implies that 

E|L4/2|| 2 <n(l + o(l)). (11) 
Combining P), (JTDJ) and (PJJ, it follows that 



E 



A 



< v / n\/a 2 + 5/4 + o(l). 



This completes the proof of the first part of Theorem SJ Now, we describe a tail bound for a 
convex function of random signs. This result was established by Ledoux [19]; see also [201 § 5.2] for 
a discussion of concentration in product spaces. 

Theorem 10 Suppose f : R m — > R is a convex function that satisfies the Lipschitz bound 

\f(x)-f(y)\<L\\x-y\\ for all x,y G R m 
Let r G {±l} m be a random sign vector. For all e > 0, 

Pr [f(r) > E/(r) + Le] < exp(-e 2 /8). 

Let / : R n ( n_1 )/ 2 — y R. Consider the bijection between the set of symmetric matrices of size n 
having zero diagonal and the set of vectors in R™^" 1 )/ 2 via the natural vectorization function 
vec : R nxn — > R n ( n ~ 1 )/ 2 , i.e., vectorize the strict lower triangular part of its argument matrix. 
For every vector v G R n ( n_1 )/ 2 , define f(v) : = ^vec _1 (u) + ^7= J • By definition, / is convex. 
Let's compute its Lipschitz constant; let A, B be symmetric matrices of size n with zero diagonals, 
then |/(vec(A))-/(vec(S))| = 1 1| \A + ^j|| - \\B + ^Lj| 
1 1 vec (-4) — vec(B)|| where ||-A||p = Ylij-^-lj (* ne inequality \\A 

Lipschitz constant of / is one, which concludes the proof of the "moreover" part of Theorem 01 
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